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Abstract 

Background: The identification of eligible controlled trials for systematic reviews of complementary and alternative 
medicine (CAM) interventions can be difficult. To increase access to these difficult to locate trials, the Cochrane 
Collaboration Complementary Medicine Field (CAM Field) has established a specialized register of citations of CAM 
controlled trials. The objective of this study is to describe the sources and characteristics of citations included in the 
CAM Field specialized register. 

Methods: Between 2006 and 2011, regular searches for citations of CAM trials in MEDLINE and the Cochrane 
Central Register of Controlled Trials (CENTRAL) were supplemented with contributions of controlled trial citations 
from international collaborators. The specialized register was 'frozen' for analysis in 201 1, and frequencies were 
calculated for publication date, language, journal, presence in MEDLINE, type of intervention, and type of medical 
condition. 

Results: The CAM Field specialized register increased in size from under 5,000 controlled trial citations in 2006 to 
44,840 citations in 201 1. Most citations (60%) were from 2000 or later, and the majority (71%) were reported in 
English; the next most common language was Chinese (23%). The journals with the greatest number of citations 
were CAM journals published in Chinese and non-CAM nutrition journals published in English. More than one-third 
of register citations (36%) were not indexed in MEDLINE. The most common CAM intervention type in the register 
was non-vitamin, non-mineral dietary supplements (e.g., glucosamine, fish oil) (34%), followed by Chinese herbal 
medicines (e.g., Astragalus membranaceus, Schisandra chinensis) (27%). 

Conclusions: The availability of the CAM Field specialized register presents both opportunities and challenges for 
CAM systematic reviewers. While the register provides access to thousands of difficult to locate trial citations, many 
of these trials are of low quality and may overestimate treatment effects. When including these trials in systematic 
reviews, adequate analysis of their risk of bias is of utmost importance. 
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Background 

Complete identification of eligible controlled trials is an 
essential step in conducting a systematic review, and 
finding and collecting citations of controlled trials have 
been aims of the Cochrane Collaboration from its incep- 
tion 20 years ago. As part of this mission, the Collabo- 
ration developed the Cochrane Central Register of 
Controlled Trials (CENTRAL), a searchable database of 
citations of controlled trials [1]. The Collaboration has 
agreements with the publishers of MEDLINE and 
EMBASE, ensuring that all citations of controlled trials 
from those databases are republished in CENTRAL. 
Cochrane entities (e.g., Cochrane Review Groups) also 
regularly submit citations of controlled trials to CEN- 
TRAL, ensuring that CENTRAL contains trial citations 
from multiple sources, including not only MEDLINE 
and EMBASE, but also regional and subject-specific 
databases, as well as trial citations not included in data- 
bases. Because of the Collaboration's extensive efforts at 
identification of trials from a range of sources, CEN- 
TRAL is considered to be the most comprehensive 
source of citations of controlled trials for inclusion in 
systematic reviews [1]. 

The complete identification of eligible controlled trials 
can be particularly challenging for systematic reviews of 
CAM interventions. The disadvantage of relying exclu- 
sively upon sources such as MEDLINE for trial identifica- 
tion is illustrated by research conducted by Egger and 
colleagues. Egger and colleagues [2] analyzed the charac- 
teristics of 1,635 controlled trials included in a group of 
n = 159 systematic reviews, including both conventional 
and CAM-related reviews. They found that in CAM- 
related systematic reviews, the proportion of non- 
MEDLINE-indexed trials (41%) was approximately twice 
that proportion seen in conventional medicine systematic 
reviews (21%). If the systematic review authors had 
searched only MEDLINE for trials, they would have 
missed many trials, possibly including some important tri- 
als, and the proportion missing from CAM reviews would 
have been double the proportion missing from conven- 
tional medicine reviews. Earlier research using various 
'gold standard sets' of known trials for specific CAM inter- 
ventions found that the percentage of known trials in- 
cluded in MEDLINE was 58% for acupuncture trials [3], 
31% for ginkgo trials [4], and 17% for homeopathy trials 
[4]. Ensuring that CENTRAL contains both MEDLINE 
and non-MEDLINE citations of CAM-related controlled 
trials is therefore important for the unbiased conduct of 
Cochrane systematic reviews of CAM interventions. 

The CAM Field maintains a specialized register of 
citations of controlled trials of CAM interventions, 
which is a 'subregister' of CENTRAL. In 1998, a biblio- 
metric analysis of the CAM Field specialized register 
described it as containing 3,774 controlled trials [5]. In 



2006, the CAM Field began an active program to im- 
prove the scope and size of the CAM Field specialized 
register. CAM Field staff and international partners in 
this endeavor performed searches of bibliographic 
databases and paper journals, and CAM Field staff 
performed extensive quality checks and de-duplication 
of all identified citations. As a result of these efforts, the 
CAM Field register contained a total of 43,310 CAM 
Field specialized register citations as of Issue 1, 2012, of 
The Cochrane Library. CAM Field specialized register 
citations can be retrieved from the Cochrane Central 
Register of Controlled Trials (CENTRAL) database of 
The Cochrane Library by searching for the tag 'SR- 
COMPMED' in all text. The objective of this study is to 
describe the sources and characteristics of the trial cita- 
tions included in the CAM Field specialized register. 

Methods 

In 2011 we drew up a detailed protocol (see Additional 
file 1) that described the eligibility criteria for including 
citations and the methods we had used to build the 
register. The protocol also pre-specified the methods we 
planned to use in examining the characteristics of the 
citations in the register. A summary of these methods, 
as well as the details of contributions of trial citations by 
international collaborators, is presented below. 

Eligibility criteria 

All citations in the CAM Field specialized register are 
required to meet the following two inclusion criteria: (1) 
they must be reports of controlled trials, and (2) they 
must be CAM-related. 

We considered controlled trials to be studies meeting 
the inclusion criteria for CENTRAL that were formu- 
lated and agreed upon in November 1992 and are pub- 
lished in Chapter 6.3 of the Cochrane Handbook [6]. We 
considered trials to be CAM-related if they described in- 
terventions that are outside the practices and theories of 
disease and healing that are intrinsic to the conventional 
Western medical model [7]. To retrieve citations of 
CAM trials from MEDLINE and CENTRAL, we relied 
upon the CAM on PubMed search strategy, which was 
jointly developed by the US National Library of Medi- 
cine and the US National Institutes of Health, National 
Center for Complementary and Alternative Medicine, 
and introduced in PubMed in 2001 [8,9]. In cases where 
some uses of the intervention are accepted within 
conventional Western medicine and others are not (e.g., 
vitamin supplementation), the CAM on PubMed search 
strategy generally does not distinguish between conven- 
tional and unconventional uses of an intervention. 
Therefore, for classifying interventions as conventional 
or CAM, we followed the same operational criteria 
we had previously developed for classifying Cochrane 
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systematic reviews as conventional or CAM [10]. Some 
of the major decisions about the scope of CAM were as 
follows: we excluded vitamins and other supplements 
that are administered parenterally in hospital settings, 
we excluded dietary supplementation for treatment or 
prevention of medically diagnosed deficiency states (e.g., 
iron supplementation for preventing or treating iron 
deficiency), and we excluded vitamin supplements for 
preventing or treating disease in countries where vitamin 
deficiency is widespread (e.g., vitamin A for treating 
measles in children in Niger). We included vitamins for 
other conditions, even vitamins that are accepted for the 
prevention or treatment of specific disorders (e.g., folic 
acid for preventing neural tube defects). In general, we 
decided that we should be over-inclusive rather than 
under-inclusive with vitamin therapies, aside from the 
three major exclusions detailed above, and therefore 
some of the vitamin trials in the database would not be 
accepted as CAM by most people. Finally, we excluded 
exercise interventions with the exception of mind-body 
exercise (e.g., yoga), and we excluded conventional psy- 
chotherapies. A full description of the CAM Field oper- 
ational definition of CAM has been published previously 
(see Additional file 2) [10]. 

Methods for building the register of trials 

We began the expansion of the CAM Field specialized 
register of trials by building upon the reference manage- 
ment database of nearly 5,000 CAM controlled trial cita- 
tions developed during the 1990s and early 2000s by 
Vickers and colleagues [5]. In 2006, we began regular 
searches of MEDLINE in PubMed using the CAM on 
PubMed search strategy. In 2008, an information spe- 
cialist translated the CAM on PubMed search strategy 
into a format for use in CENTRAL, and we replaced 
searches of PubMed with regular searches of CENTRAL. 
The rationale for replacing searches of PubMed with 
searches of CENTRAL is that CENTRAL includes not 
only controlled trial citations from MEDLINE, but also 
controlled trial citations from multiple other sources. 
These other citations, which may be from other data- 
bases or from difficult to locate sources such as trial 
proceedings, are identified by Cochrane contributors 
around the world and contributed to CENTRAL. All 
Cochrane groups then search CENTRAL in order to 
identify relevant citations that others have contributed. 
We began by searching CENTRAL from inception and 
then searched newly added citations in each subsequent 
issue of CENTRAL. 

An important subset of CAM is traditional medicine, 
defined by the World Health Organization as "the sum 
total of knowledge, skills and practices based on the the- 
ories, beliefs and experiences indigenous to different cul- 
tures that are used to maintain health, as well as to 



prevent, diagnose, improve or treat physical and mental 
illnesses. Traditional medicine that has been adopted by 
other populations (outside its indigenous culture) is 
often termed alternative or complementary medicine" 
[11]. Different countries often have their own forms of 
traditional medicine (e.g., traditional Chinese medicine, 
traditional Korean medicine). Because we did not have 
access to the traditional medicine trial reports that are 
published in regional or national databases and journals, 
in 2008 we began contacting Cochrane colleagues and 
contributors to solicit the contributions of citations of 
traditional medicine trials published in their regions. 
These efforts are described below. Searches of PubMed, 
and then of CENTRAL, were thus complemented 
with searches of bibliographic databases and journals 
conducted by several international groups who contrib- 
uted citations of trial reports to the CAM Field for in- 
clusion in the CAM Field register. Citations provided by 
contributing organizations were not restricted by publi- 
cation year. As described below, two of these contribut- 
ing groups also provided PDFs of the full text 
publications for all identified citations. Collaborators are 
listed below in order of numbers of citations submitted 
to the CAM Field for the specialized register. 

Traditional Chinese medicine (TCM) trials identified by 
Chinese collaborators 

Beginning in 2008, staff at the Center for Evidence- 
Based Medicine of the Beijing University of Chinese 
Medicine, under the direction of Jianping Liu, searched 
both electronic databases and Chinese journals to iden- 
tify citations of controlled trials of TCM interventions. 
The journal titles, article titles, and abstracts (if avail- 
able) of all identified citations were translated into 
English, entered into a reference management database 
with added topic keywords, and submitted to the CAM 
Field for inclusion in the specialized register. The full 
text report was also submitted to the CAM Field for 
each citation in a PDF format. 

Trials from CAM-specific databases identified by Canadian 
collaborators 

In 2008, information specialists under the direction of 
David Moher of the Ottawa Hospital Research Institute 
undertook a project to search several specialized data- 
bases for difficult to locate controlled trials of CAM 
interventions, and these searches were replicated in 
2010 [12]. All identified citations were imported into a 
reference management database, information about the 
source database and the type of CAM intervention was 
included for each citation, and the database was submit- 
ted to the CAM Field for inclusion in the CAM Field 
specialized register. 



Wieland et al. Systematic Reviews 2013, 2:51 
http://www.systematicreviewsjournal.eom/content/2/1/51 



Page 4 of 13 



Traditional medicine trials identified by Korean 
collaborators 

In 2010, researchers at the Korea Institute of Oriental 
Medicine, under the direction of Myeong Soo Lee, 
searched both electronic databases and journals to iden- 
tify citations of controlled trials of traditional medicine 
interventions conducted in Korea and primarily pub- 
lished in non-MEDLINE journals. Initial searches were 
focused on identifying trials of acupuncture [13] and 
ginseng, and were then expanded to include all other 
CAM interventions. The journal titles, article titles, and 
abstracts (if available) of all identified citations were 
translated into English and entered into a reference 
management database with topic keywords, and the 
citations were submitted to the CAM Field. For each 
citation, the full text publication was also submitted to 
the CAM Field in a PDF format. 

Kampo trials identified by Japanese collaborators 

Kampo is the Japanese adaptation of traditional Chinese 
medicine. While Kampo uses most of the interventions of 
Chinese medicine, including acupuncture and moxibus- 
tion, its primary focus is on the study and evaluation of 
traditional herbal medicines. In 2001, the Japan Society for 
Oriental Medicine undertook a project to collect con- 
trolled trial evidence on Kampo interventions through 
searches of both electronic databases and journals [14]. As 
randomized controlled trials of Kampo interventions are 
identified, structured abstracts are prepared for each trial, 
and the citations and structured abstracts are published 
online in English. In March 2011, one of the leaders of this 
initiative, Kiichiro Tsutani of the University of Tokyo, pro- 
vided permission for the CAM Field staff to incorporate 
the citations associated with these Kampo trials into the 
CAM Field specialized register together with links to the 
online structured abstracts. 

Ayurveda and other CAM-related trials identified by 
Indian collaborators 

The South Asian Database of Controlled Clinical Trials 
(SADCCT) is an online database of citations of controlled 
trials that have been conducted in countries for which the 
South Asian Cochrane Network & Centre is the reference 
Cochrane Centre, including Afghanistan, Bangladesh, 
Bhutan, India, the Maldives, Nepal, Pakistan, and Sri 
Lanka. The SADCCT was developed by searching South 
Asian journals and conference proceedings for all con- 
trolled trials [15]. In 2011, staff at the South Asian 
Cochrane Network & Centre, under the direction of 
Prathap Tharyan, identified and forwarded citations of 
South Asian trials of Ayurveda, Unani, Siddha, and other 
CAM interventions included in the SADCCT, and in 2012 
staff at the CAM Field identified additional trial citations 
from the online SADCCT. 



CAM trials identified by African collaborators 

The African Trials Register is a database of citations of 
controlled trials conducted in Africa. It has been deve- 
loped at the South African Cochrane Centre by the 
Cochrane Centre staff searching African journals and elec- 
tronic bibliographic databases [16,17]. In 2011, citations of 
CAM-related trials included in the African Trials Register 
were identified by Elizabeth Pienaar, information specialist 
at the South African Cochrane Centre, and a database of 
citations was forwarded to the CAM Field. 

Methods for examining the contents of the register of 
trials 

In August 2011, we suspended additions of new search 
results to the CAM Field specialized register and began 
an intensive program of cleaning and updating the regis- 
ter in preparation for analysis, focusing on detection and 
removal of any non-CAM or non-controlled trial cita- 
tions, deduplication of register citations, identification of 
whether each register citation was present or absent in 
MEDLINE, and standardization of journal names (see 
Additional file 1 for details of procedures used). Because 
EMBASE is a second major database from which con- 
trolled trials are automatically downloaded to CEN- 
TRAL, we also wished to characterize the EMBASE 
coverage of CAM Field register citations. However, 
register citations do not contain EMBASE identifiers, 
and we did not have the resources to comprehensively 
check all citations for presence in EMBASE. We there- 
fore estimated the proportion of register citations 
present in EMBASE by taking a random sample of 200 
register records and searching EMBASE for each citation 
in the sample. We used the same random sample to esti- 
mate the overlap in coverage between MEDLINE and 
EMBASE, and the proportion of register citations not 
present in either MEDLINE or EMBASE. 

We sorted journal titles by frequency and the 100 
journals associated with the greatest number of trial cita- 
tions were classified as either CAM or conventional in 
focus, using the classification method described in the 
protocol (see Additional file 1). For each of the 25 CAM 
and 25 conventional journals with the greatest number 
of trial citations, we determined the language of publica- 
tion and whether the journal was indexed in MEDLINE. 
In addition to characterizing the journals with the 
greatest number of citations, we also examined the dis- 
tribution of citations across all journals in the register to 
determine to what extent citations are scattered across 
journals overall. 

To characterize register citations by CAM interven- 
tion, we chose 21 different types of CAM interventions 
within five broad categories. We based the 5 broad cat- 
egories and the 21 intervention types within categories 
upon the CAM Field topics list for Cochrane reviews of 
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CAM interventions [10] as well as other classifications 
of CAM interventions (e.g., the classifications of CAM 
interventions used in the 2007 NHIS survey of use of 
CAM in the United States [18]). We then developed 
subject searches for each of the 21 types of CAM inter- 
ventions by parsing the 2006 translation of the CAM on 
PubMed search strategy into the CAM intervention 
topic areas. We also consulted additional sources to 
identify any supplementary terms and to help under- 
stand and delineate between the CAM intervention topic 
areas. Searches were run and tested in MEDLINE, using 
each relevant term. An information specialist developed 
the MEDLINE version of each search strategy, which 
was then peer reviewed by an independent information 
specialist using the PRESS standard [19]. Searches were 
then adapted so that they could be used to search the 
reference management database containing the register. 
MEDLINE Medical Subject Heading (MeSH) terms and 
free text terms were sought in all database fields. The 
search strategies for identifying each of the 21 different 
types of CAM interventions are included as an additional 
file so that these strategies will be publicly available to 
CAM systematic reviewers developing searches for any of 
these classes of therapies (see Additional file 3). 

To characterize register citations by medical condi- 
tions, we used the 25 categories of medical conditions 
listed in the browse list on the home page of The 
Cochrane Library [20]. We developed subject searches 
for each medical condition category by consulting 
Cochrane reviews grouped under each of these 25 
categories to ascertain relevant search terms and con- 
cepts, and additional sources to identify any supple- 
mentary terms. Searches were run and tested in 
MEDLINE in the same manner as described above 
for the CAM intervention searches, although these 
searches on medical conditions were not peer 
reviewed. Search strategies were then adapted so that 
they could be used to search the CAM register and 
augmented with additional free text synonyms for 
relevant medical conditions. Both MeSH and free text 
terms were sought in all reference management data- 
base fields. 

Results 

At the time of our analysis, the CAM Field register of 
trials included 44,840 citations of CAM trials, which 
represented approximately 6% of the total number of all 
trial citations in CENTRAL [21]. Of these 44,840 CAM 
trial citations, 15,990 (36%) are not included in 
MEDLINE. Among the random sample of 200 register 
citations checked for EMBASE status, 63/200 (31.5%; 
95% CI 25% to 38%) are not included in EMBASE. 
Among these 63 citations not included in EMBASE, only 
7/63 (11.1%; 95% CI 3% to 19%) are included in 



MEDLINE. Among the 69/200 sample citations not in- 
cluded in MEDLINE, only 13/69 (18.8%; 95% CI 10% to 
28%) are included in EMBASE. Overall, 56/200 (28%; 
95% CI 22% to 34%) of sample citations are not included 
in either MEDLINE or EMBASE. 

There is a substantial increase in the numbers of trial 
citations published for each 5-year time period, and the 
majority of the citations are from more recent publica- 
tion years (see Figure 1). This increase in the numbers 
of trial citations included in the CAM Field register over 
publication year time periods corresponds to a similar 
increase over publication year periods seen overall in 
CENTRAL [22]. 

The most common languages in the register after 
English are Chinese {n = 10,376; 23%), German (n = 963; 
2%), Korean (n = 330; 0.7%), Japanese (« = 312; 0.7%), 
and Russian (n = 227; 0.5%). The representation of cita- 
tions in languages other than English probably reflects 
both the number of trials published in that language as 
well as our methods of sourcing the trial citations for 
the register. For example, the number of trials in 
Chinese reflects both the fact that Chinese journals pub- 
lish a large number of trials of traditional Chinese medi- 
cine interventions (e.g., acupuncture, Chinese herbal 
medicine) [23] and also the fact that we have collabo- 
rated with our partner institution in China, since 2008, 
to search Chinese journals and databases to identify 
these trials. More recent collaborations have resulted in 
the identification and inclusion in the register of trad- 
itional medicine trial citations from other countries 
(see Table 1). Searches of trials by several of these collab- 
orating institutions are ongoing. 

The 25 conventional journal titles and 25 CAM jour- 
nal titles with the largest number of citations in the 
register are listed in Tables 2 and 3, respectively. Nearly 
all (24/25) of the top conventional medicine journals are 
MEDLINE-indexed, while only 12/25 of the top 25 
CAM journals are MEDLINE-indexed. Together, these 
50 journals account for 13,731 trials (30.6% of the total) 
and 8,445 of these trial citations (61.5%) are MEDLINE- 
indexed. The clinical focus of the conventional journals 
was concentrated in nutrition (n = 8), general and in- 
ternal medicine (n = 4), and pediatrics (n = 3). The clin- 
ical focus of the top 25 CAM journals was concentrated 
in TCM (n = 20). 

The CAM register contains citations from 4,845 
journals. Citations are quite concentrated in a few 
journals. One-third of the citations are found in the top 
57 journals and two-thirds come from the top 420 
journals. Among the 4,425 journals containing the 
remaining one-third of citations, 2,749/4,425 (62%) con- 
tributed only 1 or 2 citations to the register. 

Of the 44,840 trial citations in the register, 93% were 
classified into one or more of the CAM intervention 
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Before 1967-1971 1972-1976 1977-1981 1982-1986 1987-1991 1992-1996 1997-2001 2002-2006 2007-2011 
1967 

Publication year of citation (in five year groupings) 



Figure 1 Number of citations in the CAM Field specialized register by citation publication year. * The drop in the number of trial citations 
for 2007-201 1 may be partially due to incomplete indexing of trials published in this recent time period and/or may be partially due to our 
having suspended additions of new search results to the CAM Field specialized register in August 201 1. 



categories for which searches were conducted. The 
greatest concentrations were in non-vitamin, non- 
mineral dietary supplements (e.g., glucosamine, fish oil); 
Chinese herbal medicine (e.g., Astragalus membrana 
ceus, Schisandra chinensis); diet-based therapies; vitamin 
and mineral interventions; and acupuncture (Table 4). 
The high representation of acupuncture and Chinese 
herbal medicine trial citations in the register might be 
explained by both a large number of trials being pub- 
lished in these areas as well as by the methods that we 
used to source the trial citations for the database (as de- 
scribed above). 



Of the 44,840 citations in the register, 85% were classi- 
fied into 1 or more of the 25 categories of medical 
conditions. The greatest concentrations were in the cat- 
egories of heart and circulation, anesthesia and pain, 
mental health, and endocrine and metabolic conditions 
(Table 5). Categories varied greatly in the proportion of 
citations included on MEDLINE, and the lowest percent- 
age of MEDLINE-indexed citations was among citations 
not classified into any medical condition category. This 
is likely a result of non-MEDLINE citations being less 
likely to have abstracts or detailed keywords, and thus 
being less easy to categorize through searches. 



Table 1 Source of citations in the CAM Field specialized register 



Source of citations 


All citations n (% of 
citations in register) 


Non-MEDLINE N (% of non- 
MEDLINE citations in register) 


Contributing organization 






Beijing University of Chinese Medicine 


6,484 (14%) 


6,183 (39%) 


Ottawa Hospital Research Institute 


2,967 (7%) 


2,777 (17%) 


Japan Society for Oriental Medicine 


351 (1%) 


293 (2%) 


Korea Institute of Oriental Medicine 


307 (1%) 


304 (2%) 


South Asian Cochrane Centre & Network 


71 (<1%) 


31 (<1%) 


South African Cochrane Centre 


26 (<1%) 


0(0) 


Total citations from contributing organizations 


10,206 (23%) 


9,588 (60%) 


Total citations from other sources (e.g., searches of CENTRAL) 


34,634 (77%) 


6,402 (40%) 


Totals 


44,840 (100%) 


15,990 (100%) 
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Table 2 Twenty-five conventional medicine journals with the most citations in the CAM Field specialized register 



Journal name* 



Number of citations 

(% of citations in register) 



Language of full text MEDLINE-indexed 



The American Journal of Clinical Nutrition 1,027 (2.3) 

European Journal of Clinical Nutrition 381 (0.8) 

The Journal of Nutrition 362 (0.8) 

The British Journal of Nutrition 310 (0.7) 

Lancet 247 (0.6) 

BMJ 223 (0.5) 

Pain 182(0.4) 

Journal of Consulting and Clinical Psychology 178 (0.4) 

Behaviour Research and Therapy 170 (0.4) 

Journal of Clinical Rehabilitative Tissue Engineering Research 166 (0.4) 
(Zhong Guo Zu Zhi Gong Cheng Yan Jiu Yu Lin Chuang Kang Fu) 

Archives of Physical Medicine and Rehabilitation 164 (0.4) 

Journal of the American College of Nutrition 156 (0.3) 

Diabetes Care 144(0.3) 

JAMA: Journal of the American Medical Association 1 36 (0.3) 

Atherosclerosis 132(0.3) 

Journal of Clinical Endocrinology and Metabolism 131 (0.3) 

Nutrition (Burbank, Los Angeles County, Calif.) 128 (0.3) 

Pediatrics 127(0.3) 

The Journal of Pediatrics 121(0.3) 

Lipids 119(0.3) 

The New England Journal of Medicine 1 1 1 (0.2) 

Arzneimittel-Forschung 110(0.2) 

Medicine and Science in Sports and Exercise 91 (0.2) 

Journal of Pediatric Gastroenterology and Nutrition 90 (0.2) 

Journal of the American Dietetic Association 88 (0.2) 

Total 5,094(11.4) 



English 
English 
English 
English 
English 
English 
English 
English 
English 
Chinese 

English 
English 
English 
English 
English 
English 
English 
English 
English 
English 
English 

English, Germant 
English 
English 
English 



Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
No 

Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 
Yes 



•Totals include named journals and any predecessor journals that were continued by the named journal. 
tArticles are published in either German or English, and abstracts are available in both languages. 



Discussion 

The CAM Field specialized register is an important re- 
source for both MEDLINE and non-MEDLINE citations 
of CAM controlled trials. The prevalence of MEDLINE- 
indexed trial citations reflects the searches conducted in 
PubMed (and later in CENTRAL) for MEDLINE- 
indexed citations retrieved using the CAM on PubMed 
search strategy. We must therefore acknowledge the 
strides made in identification of both controlled trial 
and CAM citations by the US National Library of 
Medicine since the CAM Field specialized register was 
last examined in 1998 [5]. The large number of non- 
MEDLINE citations reflects searches of CENTRAL, 
which includes non-MEDLINE citations, and the efforts 
of CAM Field collaborators in China, Canada, Japan, 
and Korea, whose contributions to the specialized regis- 
ter were of predominantly non-MEDLINE citations. 



Overall, less than two-thirds of register citations are 
MEDLINE-indexed, and Sampson et al. concluded that 
with incomplete MEDLINE indexing of a body of litera- 
ture, a specialized register was of particular utility [24] . 

Particular strengths of the register include citations of 
nutritional and supplement-related interventions, and 
traditional medicine. While citations of chelation the- 
rapy, nutrition and supplement-related interventions, 
sensory art therapies, relaxation, and meditation are 
likely to be MEDLINE-indexed (at least 80% of all these 
citations are indexed in MEDLINE), citations of trad- 
itional medicine interventions and homeopathy are less 
likely to be MEDLINE-indexed (fewer than 50% indexed 
in MEDLINE) and thus may be more difficult to locate. 
Therefore, the CAM Field specialized register may be a 
particularly useful resource for identifying citations of 
trials to be included in systematic reviews of traditional 
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Table 3 Twenty-five complementary medicine journals with the most citations in the CAM Field specialized register 


Journal name* 


Number of citations 

(% of citations in register) 


Language of 
full text 


MEDLINE- 
indexed 


Chinese Journal of Integrated Traditional and Western Medicine 
(Zhong Guo Zhong Xi Yi Jie He Za Zhi) 


1,809 (4.0) 


Chinese 


Yes 


Chinese Journal of Information on Traditional Chinese Medicine 
(Zhong Guo Zhong Yi Yao Xin Xi Za Zhi) 


1 ,049 (2.3) 


Chinese 


No 


Chinese Acupuncture & Moxibustion (Zhongguo zhen jiu) 


1,021 (2.3) 


Chinese 


No 


Shanghai Journal of Acupuncture and Moxibustion (Shang Hai Zhen Jiu Za Zhi) 


557 (1.2) 


Chinese 


No 


Journal of Traditional Chinese Medicine 


367 (0.8) 


English 


Yes 


Journal of Alternative and Complementary Medicine 


357 (0.8) 


English 


Yes 


Chinese Journal of Integrated Traditional and Western Medicine on Liver Diseases 
(Znong xi yi j le He ban Ding za zmj 


356 (0.8) 


Chinese 


No 


Modern Journal of Integrated Traditional Chinese and Western Medicine (Xian Dai 
Zhong Xi Yi Jie He Za Zhi) 


35U I.U.O) 


Chinese 


Mrs 

NO 


Journal of Manipulative and Physiological Therapeutics 


274 (0.6) 


English 


Yes 


Chinese Traditional Patent Medicine (Zhong Cheng Yao) 


231 (0.5) 


Chinese 


No 


China Journal of Chinese Materia Medica (Zhong Guo Zhong Yao Za Zhi) 


189 (0.4) 


Chinese 


Yes 


Shanghai Journal of Traditional Chinese Medicine (Shang Hai Zhong Yi Yao Za Zhi) 


186 (0.4) 


Chinese 


No 


Hebei Journal of Traditional Chinese Medicine (He Bei Zhong Yi) 


1 QA fn A\ 
I o4 (U.4J 


Chinese 


\| 0 


Chinese Journal of Integrative Medicine 


1 7Q lr\ A\ 
1 /o (U.4J 


English 


Yes 


The American Journal of Chinese Medicine 


I by (U.4J 


English 


Yes 


Shandong Journal of Traditional Chinese Medicine (Shan Dong Zhong Yi Za Zhi) 


163 (0.4) 


Chinese 


No 


Complementary Therapies in Medicine 


1 ^n 3\ 

I Z>U {U.Dj 


English 


I cS 


World Journal of Acupuncture-Moxibustion 


1/17 /<~\ 3\ 
14/ [U.Dj 


English 


M/-i 
\l 0 


China Journal of Traditional Chinese Medicine and Pharmacy 
(Zhong Hua Zhong Yi Yao Za Zhi) 


146 (0.3) 


Chinese 


No 


JldllySU JUUIIIdl Ul lldUIUUIId V—lllllcSt: IVICUILI [ lc ^Jldliy JU LA lUI ly II iduy 


1 34 (0 3) 


Chinese 


No 


Acupuncture Research (Zhen Ci Yan Jiu) 


1 33 (0.3) 


Chinese 


Yes 


Journal of Chinese Integrative Medicine (Zhong Xi Yi Jie He Xue Bao) 


129 (0.3) 


Chinese 


Yes 


Phytomedicine: International Journal of Phytotherapy and Phytopharmacology 


1 28 (0.3) 


English 


Yes 


Journal of Psychosomatic Research 


1 24 (0.3) 


English 


Yes 


Journal of the Korean Acupuncture & Moxibustion Society 
fTaehan Chimgu Hakhoe chi) 


1 06 (0.2) 


Korean 


No 


Total 


8,637 (19.3) 







•Totals include named journals and any predecessor journals that were continued by the named journal. 



medicine interventions, particularly TCM. Similarly, the 
CAM Field specialized register may be a useful source of 
trials for systematic reviews covering CAM interventions 
for medical conditions in which a lower proportion of 
citations are MEDLINE-indexed (e.g., endocrine and 
metabolic conditions or infectious disease). 

Among the 4,845 journals containing citations in the 
CAM Field register, 9% contain two-thirds of the register 
citations, and 57% contain only one or two register cita- 
tions each. One unanswered question is whether this 
distribution of journals in the register represents the true 
distribution of CAM trials across journals or whether in- 
stead it is an artifact resulting from the way that the 



register was developed. There is no way to definitively 
answer this question because there exists no 'gold stand- 
ard' complete database of CAM trials against which the 
journal distribution in the CAM Field register can be 
compared. However, we believe that the distribution of 
journals in the CAM Field register is largely an artifact 
of the way the register was developed. This is because, 
in identifying citations for register inclusion, contribu- 
tors often searched bibliographic databases to identify 
trial citations on specific topics for their systematic re- 
views rather than comprehensively searching entire 
journals for all CAM trials. As a result, some journals 
for which only one or two CAM trial citations were 
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Table 4 Number of CAM Field specialized register citations classified by type of CAM interventions 


CAM intervention 


Citations n (% of 
citations in register) 


MEDLINE-indexed citations N (% of 
citations in intervention category 
that are MEDLINE-indexed) 


Non-vitamin, non-mineral dietary supplements 
(e.g., glucosamine, fish oil) 


15,140 (33.8) 


1 2,529 (82.8) 


Chinese herbal medicine (e.g., Astragalus membranaceus, 
Schisandra chinensis) 


12,118 (27.0) 


3,575 (29.5) 


Diet-based therapies 


9,009 (20. 1) 


9,009 (88.3) 


Vitamin and mineral interventions (includes megavitamin 
therapies and vitamin or mineral therapies for other than 
medically diagnosed deficiencies or deficiency-related disorders) 


~7~7/l1 /1 "7 n\ 

7,741 (17.3) 


6,468 (83.6) 


Acupuncture 


6,035 (13.5) 


2,632 (43.6) 


Relaxation (includes guided imagery and deep breathing) 


3,743 (8.3) 


3,194 (85.3) 


nterventions using veritable energy modalities (unconventional 
uses of magnets, phototherapy, electrical stimulation, 
or ultrasonic therapy) 


2,977 (6.6) 


2,265 (76.1) 


Chiropractic or osteopathic manipulation 


2,606 (5.8) 


2,041 (78.3) 


Biofeedback 


2,109 (4.7) 


1 ,643 (77.9) 


Massage 


1,481 (3.3) 


987 (66.6) 


Traditional medicine not otherwise specified (e.g., Ayurveda, Kampo) 


1,409 (3.1) 


560 (39.7) 


Meditation (includes mindfulness-based therapies) 


1 ,259 (2.8) 


1 ,056 (83.9) 


n ■ 1*111 .J ■ 4. j_ 1 4_j_i ■ ■r*i 

Biologically based interventions not otherwise specified 

(e.g., balneotherapy, prolotherapy) and excluding interventions 

using energy fields 


1,215 (2.8) 


944 (77.7) 


Interventions using putative energy fields (distant healing, 
prayer, qi gong, reiki, spiritual healing, and therapeutic touch) 


1,210 (2.7) 


801 (66.2) 


Sensory art therapies (includes art, dance, drama, music, 
and play therapy) 


1,136 (2.5) 


911 (80.2) 


Hypnosis 


/oU { I ,/J 


ouu (,/o.yj 


nun leupd LI iy 




309 (AO Ci\ 
jUZ [HKJ.U) 


Manipulative and body-based therapies not otherwise 
specified (e.g., Alexander technique, Pilates) 


438 (0.9) 


212 (48.4) 


Yoga 


333 (0.7) 


242 (72.7) 


Tai chi 


1 88 (04) 


149 (79.3) 


Chelation therapy 


153 (03) 


148 (96.7) 


Unclassified 


3,093 (6.9) 


2,124 (68.7) 


Totals* 


44,840 


28,850 (64.3) 



*Because individual citations were frequently classified into more than one CAM intervention category, the sum of the numbers of citations classified into each 
CAM intervention category exceeds the total number of citations in the CAM Field register (i.e., 44,840). 



identified for register inclusion may have many add- 
itional trials that have not yet been identified. Continued 
efforts to identify trials will likely change the distribution 
of citations across journals. Such efforts may also change 
the characteristics of the register in other ways (e.g., the 
number and proportion of trials covering particular 
CAM interventions or published in particular languages) 
that are not possible to quantify in advance. While the 
ultimate aim is for the CAM Field register to be a com- 
prehensive source of CAM controlled trials, the register 
cannot currently be considered to be comprehensive. 
Therefore, systematic reviewers of CAM interventions 
should search multiple electronic and other sources for 



relevant CAM trials, in addition to searching the CAM 
Field register. 

The strength of the register in terms of its coverage of 
difficult to locate trials may, however, be associated with 
potential weaknesses in terms of the quality of these tri- 
als. The largest subset of non-MEDLINE citations in the 
register (51%) is trials published in Chinese. These 
Chinese-language trials were included in the register if 
the trial publication stated that a random or quasi- 
random procedure was used to assign participants to 
treatment groups. However, a recent telephone survey of 
authors of 'claimed' randomized trials conducted in 
China discovered that only 7% could be confirmed to 
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Table 5 Number of CAM Field specialized register citations classified by type of medical condition 



Medical conditions Citations n (% of MEDLINE-indexed citations N (% of citations 

citations in register) in medical condition category that are 

MEDLINE-indexed) 



Heart and circu ation 


8,028 (1 7.9) 


5,585 (69.6) 


Anesthesia and pain control 


7,656 (17.1) 


5,492 (71.7) 


Mental health 


7,472 (16.7) 


5,646 (75.6) 


Endocrine and metabolic 


6,188 (13.8) 


3,797 (61.4) 


Gastroenterology 


3,378 (7.5) 


2,354 (69.7) 


Orthopedics and trauma 


3,331 (7.4) 


2,263 (67.9) 


Cancer 


2,983 (6.7) 


2,205 (73.9) 


Lungs and airways 


2,545 (5.7) 


1,828 (71.8) 


Tobacco, drugs, and alcohol dependence 


2,286 (5.1) 


1,935 (84.6) 


Neonatal care 


2,252 (5.0) 


1,924 (85.4) 


Rheumatology 


2,199 (4.9) 


1,580 (71.9) 


nfectious disease 


2,080 (4.6) 


1,128 (54.2) 


Pregnancy and childbirth 


2,050 (4.6) 


1,642 (80.1) 


Kidney disease 


1,849 (4.1) 


1,207 (65.3) 


Neuro ogy 


1,697 (3.8) 


1,289 (76.0) 


Gynecology 


1,598 (3.6) 


1,142 (71.5) 


Skin 


1,187 (2.6) 


746 (62.8) 


Dentistry and oral health 


1,123 (2.5) 


896 (79.8) 


Ear, nose, and throat 


1,108 (2.5) 


742 (67.0) 


Eyes and vision 


764 (1.7) 


505 (66.1) 


Urology 


690 (1.5) 


593 (85.9) 


Wounds 


667 (1.5) 


505 (75.7) 


Developmental, psychosocial, and learning problems 


656 (1.5) 


567 (86.4) 


Blood disorders 


596 (1.3) 


435 (73.0) 


Genetic disorders 


214 (0.5) 


148 (69.2) 


Unclassified 


6,635 (14.8) 


3,298 (49.7) 


Totals* 


44,840 


28,850 (64.3) 



*Because individual citations were frequently classified into more than one medical condition category, the sum of the numbers of citations classified into each 
medical condition category exceeds the total number of citations in the CAM Field register (i.e., 44,840). 



use a random method to assign participants to treatment 
groups [25]. Inclusion in systematic reviews of such 
Chinese trials claiming to be randomized, but not con- 
firmed as such by systematic reviewers, may inflate these 
reviews' meta-analytic effect estimates [26]. In addition, 
a 1998 review of the outcomes of non-English language 
trials by Vickers et al. found that acupuncture trials 
conducted in China reported positive results 100% of 
the time, and Chinese trials of other interventions 
reported positive results 99% of the time, strongly 
suggesting the preferential publication in China of trials 
with positive results [27]. Although Chinese language 
trials reflect the majority of non-English language trials 
included in the register, the issue of a publication or 
reporting bias favoring positive results may also be rele- 
vant to other non-English trials included in the register. 



For example, the Vickers et al. 1998 review found 
that not only acupuncture trials from China, but also 
acupuncture trials from Japan, Hong Kong, and 
Taiwan were uniformly positive. Also, a recent pre- 
liminary investigation into the results of the Japanese 
Kampo trials included in the CAM Field register 
found that only a small number of Kampo trials have 
negative results (Kiichiro Tsutani, personal communi- 
cation, 14 March 2011), and an informal assessment 
of CAM Korean trials indicated that negative results 
were rare (Byeung-Cheul Shin, personal communica- 
tion, 13 July 2012). The positive results of non- 
English language CAM trials likely explain why 
including such trials in CAM-related systematic re- 
views tends to inflate meta-analysis effect estimates, 
according to a 2005 empirical study [28]. 
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It is possible that the methodological quality of the 
more recent Chinese-language trials may, however, be 
better than that of the earlier trials because, for example, 
the CONSORT statement has recently been more widely 
disseminated in China [29], including in Chinese 
journals of TCM [30]. In addition, while the Vickers 
et al. 1998 review [27] found that Chinese acupuncture 
trials published up to 1998 were uniformly positive, it is 
not known whether or not more recent Chinese trials 
are also likely to be uniformly positive. An important 
topic area for future research is to determine whether 
there is a publication bias favoring positive results in 
more recent trials from China. However, while 
conducting research studies to assess for the likelihood 
of publication bias in Chinese trials may be informative 
in determining the scope of the problem, the only way 
to avoid publication bias in Chinese trials is to ensure 
that all initiated Chinese language trials are known about 
through the registration of Chinese trials at inception, 
which is currently being implemented [31]. Universal 
trial registration, in conjunction with reporting of trial 
registration numbers in publications, might also serve as 
a tool in addressing duplicate publication [32], which 
some studies have observed to be prevalent among 
Chinese, Japanese, and Korean trials [33-35]. In the in- 
terim, for systematic reviews including a large number 
of Chinese trials, a possible approach for assessing the 
impact of a potential publication bias related to the 
Chinese trials may be to mark the Chinese trials in fun- 
nel plots in systematic reviews. 

Despite these concerns over the validity of Chinese tri- 
als in general, it seems inappropriate to exclude trials 
from systematic reviews on the basis of language or 
country of publication alone. A more measured ap- 
proach may be to search for Chinese trials and to tele- 
phone interview the authors of potentially eligible trials 
to try and assess whether the trials were truly random- 
ized before including them in the review. If concerns 
about the validity of the trials remain, even after the 
telephone interviews with the authors, a possible ap- 
proach is to include in the review those trials (Chinese 
or Western) for which there remains uncertainty about 
whether true randomization was used, but to be more 
restrictive when presenting the key findings, such as the 
abstract conclusions and the summary of findings table 
[36]. Another approach may be to analyze the potential 
influence of risk of bias measures (e.g., adequacy of 
randomization) on effect estimates using subgroup ana- 
lyses or sensitivity analyses. Either way, such assessment 
and analysis approaches should probably be based on 
risk of bias measures rather than on the language of 
country of origin of the trials. This is because generaliz- 
ing about individual Chinese language trials, for ex- 
ample, based on meta-research of the characteristics of 



Chinese trials overall would be an erroneous oversimplifi- 
cation. Instead, each trial included in a systematic review 
needs to be individually evaluated on its risks of bias, as- 
suming either that the trial publication is sufficiently in- 
formative or that the trial author can be contacted for 
further information. Such risk of bias assessments can 
then be incorporated into the review's analysis. 

In addition to providing a source of trials for inclusion 
in systematic reviews, the register may also be used for 
investigations into the optimal use of CAM research re- 
sources and the prioritization of future CAM reviews. 
This analysis of the types of CAM interventions and 
health conditions covered in the register is a first step in 
conducting such investigations. That is, the number of 
citations related to various CAM interventions (e.g., 
diet-based therapies) and the number of citations related 
to various medical conditions (e.g., endocrine and meta- 
bolic disorders) may be used for research into whether 
there is a correlation between those CAM interventions 
most frequently investigated in trials and those most 
commonly used, and whether the most serious or preva- 
lent health conditions are proportionately represented 
with the highest number of CAM trials. If trials of com- 
monly used interventions and/or trials for serious or 
prevalent health conditions are lacking, this may indicate 
that CAM research resources should be directed to these 
areas. In addition, the trial database may also be useful 
for prioritizing future Cochrane reviews by identifying 
CAM intervention/health condition pairings for which 
there are available trials in the register but no existing 
Cochrane review. The fact that the same classification cat- 
egories were used for CAM intervention types and health 
conditions, for both the trial citations in the specialized 
register and for a separate database of CAM-related 
Cochrane reviews [10], should facilitate such identifica- 
tion. However identifying potential future systematic re- 
views to prepare will require additional narrowing down 
of some of our CAM intervention type categories (e.g., 
"Chinese herbal medicine") and health condition categor- 
ies (e.g., "mental health") in order to identify more specific 
intervention/condition pairings (e.g., the Chinese herbal 
medicine formula Free and Easy Wanderer for depression) 
for systematic reviews. Future plans for the CAM Field 
specialized register include augmenting the size and scope 
of the register through ongoing searches and international 
partnerships, and developing methods to characterize 
groups of trials according to intervention/condition char- 
acteristics and mapping these groups to gaps in Cochrane 
systematic reviews. 

Conclusions 

The number of citations included in the CAM Field spe- 
cialized register increased nearly tenfold between 2006 
and 2011 as a result of a program of extensive searching 
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and partnerships with international collaborators. Many 
CAM Field register citations are not MEDLINE-indexed 
and many of these non-MEDLINE-indexed citations are 
published in languages other than English. While the 
register provides access to thousands of difficult to 
locate citations of trials, many of these trials are likely to 
be of low quality and may overestimate treatment 
effects. When these trials are considered for inclusion in 
systematic reviews, it is extremely important that their 
risk of bias is adequately assessed. 
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